Orbital order of spinless fermions near an optical Feshbach resonance 
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We study the quantum phases of a three-color Hubbard model that arises in the dynamics of the 
p-band orbitals of spinless fermions in an optical lattice. Strong, color-dependent interactions are 
induced by an optical Feshbach resonance. Starting from the microscopic scattering properties of 
ultracold atoms, we derive the orbital exchange constants at 1/3 filling on the cubic optical lattice. 
Using this, we compute the phase diagram in a Gutzwiller ansatz. We find novel phases with 'axial 
orbital order' in which pz and px + ipy (or p^ — ipy) orbitals alternate. 

PACS numbers: 03.75.Ss,05.30.Fk,67.85.-d,71.10.Fd 



Orbital physics of electrons plays an important role in 
strongly-correlated solid-state systems, e.g., transition- 
metal oxides (see, e.g., [1111] and references therein). In 
particular, intriguing quantum phases emerge due to the 
coupling of the orbital degree of freedom to the charge, 
spin, or lattice degrees of freedom [31 0]. Such cou- 
pling, while leading to interesting effects, also compli- 
cates the theoretical treatment. It is, therefore, desir- 
able to study simpler systems with the orbital degree of 
freedom decoupled from all others. Ultracold atoms in 
higher bands of optical lattices provide an ideal tool to 
study orbital dynamics in a well-controlled environment, 
including orbital-only models of single-species (spinless) 
fermions. 

Several groups have now achieved loading and manip- 
ulating ultracold atoms in higher (such as p-) bands of 
optical lattices [SHU- Techniques such as lattice ramp- 
ing or radio-frequency pulses have been used to transfer 
atoms from the s- to higher bands, where they can stay 
in a metastable state for a sufficiently long time. For 
spinless fermionic atoms, the p-band can also be simply 
populated by first completely filling the s-band, requir- 
ing larger particle numbers, but less experimental con- 
trol. To avoid undesired collisions between ground and 
excited-band atoms, the s-band atoms may be removed 
afterwards using laser pulses |10| . 

The interaction between fermionic atoms is usually 
weak at low temperatures because the Pauli exclusion 
principle only allows scattering in high partial-wave chan- 
nels (p, /, etc.). One way to increase the p-wave elastic 
scattering cross section is to employ a Feshbach resonance 
(FR) [TT|. Typically, this is done by coupling channels in 
the electronic ground state through magnetic fields. For 
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the case of p-waves, however, this method usually leads 
to significant atom losses through three-body inelastic 
collisions because the scattering state is well localized by 
the angular momentum barrier, and has good Franck- 
Condon overlap with more deeply bound molecules [12J. 
To circumvent this problem, recently Ref. |13| considered 
enhanced p-wave interactions via an optical FR (OFR) 
between a scattering state and an electronically excited 
"purely-long-range" molecule. Such molecules have inner 
turning points at very large distances (e.g., > 50ao in 
^^^Yb), well beyond the chemical binding region, and 
thus three-body recombination should be highly sup- 
pressed. This approach not only allows to study strongly- 
correlated phases, but also provides for a high degree of 
control. In particular, the interaction strength among 
different p-orbitals can be tuned differently. 

Motivated by these developments, we investigate in 
this article the phase diagram of spinless fermions on 
a cubic lattice near an OFR described by the following 
Hubbard-like model: 

^ = - X! ^P:i'(cjx,iCM,i+e„ + h.c.) + ^ [Vin.j,^iny^i 

+V2(na;,i«z,i + ny^n^^i) + {iVscl ^Cy^in^^ + h.c.)] . (1) 

The operator c^.i destroys a fermion in the orbital p^ at 
site i, and n^^i is the corresponding number operator. 
The lattice spacing is set to 1, e^, is the unit vector in 
direction v, and fi,i> = x, y, z. The nearest-neighbor hop- 
ping amplitude tp,^^ describes hopping of fermions in or- 
bital Pp, along the direction e^. Due to the anisotropy of 
the p-orbital Wannier wave functions, it is direction and 
orbital dependent [T1HT5]. t^^y = + (1 — b^^y). 

The interactions Vi 2,3 are induced by an OFR laser 
[T3] which couples the electronic ground state of the 
atom to an excited state. The interaction can be 
expressed in terms of the (p-wave) pseudo-potential 
V3" for two particles with mass M and relative an- 
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gular momentum to, {r) = lims-»o §^ ^^^3 d^r^. 
The real part of the p-wave scattering volume, R = 

Re[(aJ7)'^], can be tuned by the detuning and the in- 
tensity of the OFR laser. Expanding field opera- 
tors in the Wannier basis, -0 ('") = J2i i_i '^i^ ~ C/i.i, 
the interaction term J d'^ri / d^r2V'^('"i)V'^(''2)V^™('"i — 
r2)'ip{ri)il){r2) leads to the on-site, inter-orbital inter- 
action iJint = Li ^A«,i',M',i^'c).',ic|^',iC^,iC,y,i, where re- 
peated indices are summed over. (We neglect all off- 
site interactions.) The matrix element V^^^u^f^' y = 
Em / d^'^i / d^r2W^' {ri - i) w^' (r2 - i) (ri - rj) 
Wfj, (ri — i) Wu (r2 — i) can now be computed by sepa- 
rating the relative and center-of-mass coordinates. For 
deep lattices, the p-orbital Wannier functions are well 
approximated by the first excited states of harmonic os- 
cillators (with the oscillator length C controlled by the 
lattice depth). The only non-zero interaction terms are 
the ones given in Eq. ([T]), with Vi = \{Ui + U-i), 
1^2 = I {Ui + U-i + 2Uo), and = | (C/_i - Ui). Here, 
Um = SV^R/ iV^C^ M) defines the interaction strength 
in the scattering channel with angular momentum to = 
1, 0, —1. A Zeemann splitting, which may be introduced 
by a magnetic field, leads to different detuning of the 
OFR laser for the three scattering channels. This makes 
the scattering length a™ dependent on to, and conse- 
quently the J7m's can be different in magnitude and even 
in sign. Thus, the relative strengths and signs of 1^1,2,3 
can be varied by changing the strength of the Zeemann 
splitting together with the detuning of the OFR laser. By 
contrast, in a standard magnetic FR, U-i — U+i. In our 
case, breaking the symmetry between U-i and U+i leads 
to the orbital-changing term V3. Physically, it allows {px 
or py) particles to move on the two dimensional plane, 
instead of along a chain only. Since it explicitly breaks 
time-reversal symmetry (TRS), we can expect it to lead 
to novel phases reflecting that intriguing property. 

Haniiltonian ([TJ generalizes the models of Refs. |17l - 
122] . For Vi = V2, and V3 = 0, it reduces to the SU(3) 
Hubbard model. One can visualize p-band fermions as 
particles carrying a color index representing the px, Py, 
and Pz orbital states. Then, Hamiltonian ([T| describes 
a three-color fermion model with color-dependent inter- 
action, a novel color-changing term V3, and spatially 
anisotropic and color-dependent tunneling. We will show 
below that this model has a rich phase diagram with novel 
phases. Here, we focus on the strong-coupling limit for 
p-band filling 1/3, and determine the orbital order using 
a Gutzwiller mean-field ansatz. 

In the strong-coupling limit, 

|i|l|<V"i, |i||| < V"2-V^3, and \^\ <^ V2 + V3 , (2) 

double occupancy of the same site is suppressed. At 1/3 
filling of the p-band, there is on average one p-band par- 
ticle per site, and density fluctuations are frozen. Virtual 
hopping induces exchange interactions between nearest- 
neighbor orbitals (see Fig. [ij. The situation bears some 
resemblance to the emergence of magnetic models, such 
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Figure 1: Sketch of the virtual hopping processes leading to 
the effective Hamiltonian ([3|. Neglecting t_L, these - plus the 
ones obtained by interchanging x and y - axe the only ones. 
Note in particular the orbital-changing process J3. Gray ovals 
denote sites, the blue t tunneling processes, and the green 
fractions denote interactions. Orbitals are abbreviated as 



as the Heisenberg model, in the strong-coupling limit of 
the Hubbard model. The difference here is that three 
orbital (instead of two spin) states are involved. Since 
\t±\ 1 1 1 1 1 , perpendicular tunneling tj_ can safely be 
neglected |tn|, and, for brevity, we write t — t\\. Treat- 
ing the tunneling t in ([ij as a perturbation and following 
standard second-order perturbation theory, we obtain the 
effective Hamiltonian for 1/3 flUing 

i fi=x,y,z 6=±e,, 

+ X! X! (^2 - Ji)nfj,^inz,i+s 

- X! J'ii^<^\,i^v,i'^^.i+s + h-c.) , (3) 



where we have used the constraint Ux^i + ny_i + tIza — 1, 
and deflned Ji = ^ /Vi, J2 = t^V2/{Vi - Vi), J3 = 
eV^/iVi - vi), and Jx^Jy = Ji, Jz = .h- For V3 = 0, 
Vi = V2, Eq. ([3| reduces to Jf^n^ in^ ij^s, a hallmark of 
the quantum 3-state Potts-like model [23 . 

To see which orbital order is favored, we first discuss 
the simple case of J3 = 0. The first term of Eq. ^ 
always favors configurations where the orbitals at neigh- 
boring sites differ. (A) For Ji > max(J2,0), both the 
first and second terms favor an alternating pattern be- 
tween px- and py-particles in the xy-plane. (B) For 
J2 > max(Ji,0), the favored configuration is an alter- 
nating pattern between pz and not-p^. (C) For (the un- 
stable case) Ji, J2 < 0, the best configuration is a homo- 
geneously filled lattice. 

Certain aspects of Hamiltonian ([3| become clearer 
when we rewrite it in terms of the generators of the 
SU(3) group. In terms of the Gell-Mann matrices A'*^ 
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and the so-called F-spin operators Y = -^c^^\^;vCv and 
T(") = \cl\]^U^ {a = 1,2,3), ffoff becomes 

i 

^^T^^Y.^s+'^Y-r^s) (4) 

where we neglected constant terms. In the basis 
{Px,Py,Pz), Y and T^'^-' are diagonal, which means 
that terms such as YiYj, YiT^^\ or T^^^Tf^ are Ising- 

like. The orbital-changing term V3 leads to T*^^-' = 
^ _ t(-)), where r(±) are ladder operators of the 

T-spin. T*^^) and T'^^^ do not commute, but both com- 
mute with Y . This means that one can replace Y by 
its eigenvalues — | (for \pz)) and | (for jp^;) and \py)), 
which gives some insight into the physics of Hamiltonian 
^ . Assuming that the ground state is bipartite with re- 
spect to the eigenvalue of Y [24 , there are three different 
cases: (A) at all sites the eigenvalue of F is 1, (B) the 
eigenvalues — | and | alternate, and (C) all sites have 
eigenvalue — |. In the last case, there is one |p2)-particle 
per site, whence there is no virtual tunneling, and the 
Hamiltonian vanishes. In the sectors A and B, it reads 
(neglecting constant terms) 

<^ - yE E M 

Here, a denotes the usual Pauli matrices, which act on 
the subspace spanned by \px) and \py). Sector A is re- 
duced to the Ising model on decoupled xy-planes, which 
favors an antiferromagnetic ground state. This is just the 
model found in the 2D-case treated in |19l [20] . In sector 
B, VL denotes the partition where Y has eigenvalue |. On 
these sites, J3 acts as a magnetic field in the y-direction, 
lifting the degeneracy between \px) and \py) and leading 
to the ground state {\px) ± i \Py)) l^f^ (for J3 ^ 0). 

Having obtained a qualitative picture of the expected 
phases, we now analyze the phase diagram of Hamilto- 
nian ([4]) quantitatively. To this, we assume that cor- 
relations between sites are small so that the ground 
state can be approximated by a product over sites. 
To find the ground state of Hamiltonian Q, we em- 
ploy the Gutzwiller variational wave function j^*) = 
{g)j(cos6' -I- sin6'cos(/)|py)^ -I- sin6'sin(/) which 
is a product over sites i, and minimize the energy of a 
cube with side length L (up to L = 8) under periodic 
boundary conditions. Note, however, that close to phase 




Figure 2: Left: The phase diagram of HcS [Eq. (|3|] at 1/3 
filling shows four phases: (A) antiferro-orbital order (empty 
region), (B+) axial orbital order (red region, J3 > 0) and 
similarly (B_) (orange region, J3 < 0), and finally (C) with 
tunneling completely frozen (blue region). The gray wedge 
indicates the region satisfying the strong-coupling conditions 
([2|, < Ji,2 <C 1, J3 < Ji- Right: sketch of phase B+, 
in which Ip^) and \'p^) + i \py) orbitals alternate, and phase 
A. Phase B_ can be visualized from phase B+ by replacing 
\Px) + i \Vy) with |p^) - i \'Py). 



transitions, where fluctuations become important, such a 
mean- field ansatz is not valid. 

The energy per site for even L is smaller than for odd 
L, showing that the ground state periodicity is indeed 2 
|25| . In agreement with the qualitative picture above, we 
find three classes of ground states with different orbital 
order (summarized in Fig. [2|: (A) For J\ > J2 -I- IJ3I /2 
and Jl > we find an 'antiferromagnetic phase' simi- 
lar to, e.g., the 2D-model of Ref . |19j: in each a;?;-plane, 
sites with p,^.- and Py-orbitals alternate (similar to the 
antiferromagnetic Neel state) . Since Px- and pj,-particles 
do not tunnel in the z-direction, the xy-planes are de- 
coupled, and within our approximation (e.i?., neglecting 
<j^), there is no long-range order in the ^-direction. It 
is possible, however, that long-range order among the 
planes develops at low temperature for finite t^. (B) For 
Jl < J2 + I J3 1 /2 and J2 > — | J3 1 /2 the ground state 
shows axial orbital order. The state is bipartite with \pz) 
on one sublattice and {\px) ± i \Py}) /V^ (for J3 ^ 0, re- 
spectively) on the other sublattice (right-hand panel of 
Fig. [2]). The degeneracy between \px) and \py) is lifted 
by a finite J3. The state {\px) ± i \Py)) /V^ has finite an- 
gular momentum, this novel phase breaks TRS |26j . (C) 
For Jl < and J2 < — | J3I /2 Pauli exclusion prohibits 
all tunneling ty (by filling a/S-planes (a/3 — xy, xz, yz) 
uniformly with pa or p^). This state is unstable, how- 
ever, because it cannot fulfill the strong-coupling require- 
ments ([2]). Interestingly, phases A and C preserve TRS, 
although V3 in Hamiltonian ([T]) breaks it explicitly. 

Experimentally, the different phases can, e.g., 
be distinguished by measuring the density distri- 
bution after a time of flight itof- This relates to 
the in-trap momentum distribution via {n{r))^^ ^ = 
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Figure 3: Predicted time-of-flight (TOF) density distribu- 
tions, allowing to distinguish phases A and B in experiment. 
Lower (upper) row: (n (t'))^ ^ integrated along z [x) in arbi- 
trary scale. For example, w{ien viewed along the z-direction, 
phase A displays a doughnut form (lower left panel) because 
of an incoherent addition of px- and Py-Wannier envelopes. 
In phase B, the sites occupied by {\px) ±i|pj,)) /\/2 give a 
similar doughnut structure, but the hole aX kx = ky = Q is 
filled by the other half of the sites with -particles. Sim- 
ilarly, viewing along the x-direction reveals the existence of 
Pz-particles in phase B, contrary to phase A (upper row). 

[M/{ht,,,)f E^,. < (fc) (fe) (4 (fc) c. (fe)), with 

(fe) the Fourier transform of the Wannier orbital 

(r), c^(fc) = Eie'-'cM,i/i'/', and k = Mr/{hUoi). 
fc is fc modulo reciprocal lattice vectors. Features in the 
density distribution appear because of its non-trivial 
p-orbital Wannier envelope. This allows to distinguish 
phases A and B by their column density {i.e., the density 
integrated along one spatial direction), see Fig.jsj 

Observation of these novel phases requires that we si- 
multaneously achieve strong interactions, V ^ t and low 
temperatures ksT ^ /V , for the characteristic tunnel- 
ing rate t and interaction energy V . At experimentally 
feasible temperatures, this requires a significant enhance- 
ment of the real part of the p-wave scattering volume via 
the OFR. In practice, however, this is limited by sponta- 
neous emission, which broadens the resonance and also 
leads to recoil heating. For the example considered in [13] 



based on the ^Sq ^ ^Pi intercombination line in ^^^Yb, 
the atomic linewidth is sal 80 kHz, which limits the useful 
OFR p-wave enhancement. Other species such as ®''Sr, 
where the same transition has a linewidth of Ki7.5 kHz, 
should result in a substantial OFR, with a reasonable 
linewidth. Experimental studies of OFRs in related iso- 
topes are currently underway \27\ . 

In summary, we investigated the orbital order of spin- 
less fermions in the p-band of a cubic lattice with inter- 
action controlled by an OFR. The system can be realized 
with current technology. The model Hamiltonian can 
be expressed elegantly by Gell-Mann matrices. We ana- 
lyzed the orbital order in the strong-coupling limit at p- 
band filling 1/3 using a Gutz wilier- type ansatz. Besides 
a phase where all tunneling is blocked and an antiferro- 
orbital phase where Px- and pj^-orbitals alternate, we 
found a novel phase with axial orbital order which not 
only breaks translational symmetry but also has macro- 
scopic orbital angular momentum. We expect our results 
to stimulate future work on this subject. For example, it 
is interesting to investigate how quantum fiuctuations af- 
fect the phase diagram: they might distort it [25] or even 
lead to disordered 'orbital liquid' states. Fluctuations 
are also expected to lift the degeneracy between p^- and 
Py orbitals at J3 — 0, and possibly lead to spontaneous 
TRS breaking. Moreover, phase B± may have interest- 
ing topological properties. For example, at an interface 
of two domains with p^ + ipy and p^ — ipy order, chi- 
ral zero mode fermions may arise. Finally, other lattices 
and the limit of small interactions, where related models 
show non-trivial color-superfiuidity [171 [T51 [51] , are also 
interesting. 
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